Anti-Item for Deletion of Content In A Distributed Datastore

ABSTRACT

The invention provides an efficient apparatus and method for deleting items within a distributed datastore. Deletion of items is accomplished using anti-items having the same structure as a conventional data item, but for a single tag, flag, or bit which indicates that it is an anti-item, wherein the single tag, flag, or bit but does not affect the location at which a data item or anti-item is ultimately stored within the datastore, thereby ensuring that, upon insertion, the anti-item reaches the same location or locations within the datastore as the original data item.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/208,225, filed Aug. 11, 2011 which is a continuation of U.S. patentapplication Ser. No. 12/815,156, filed Jun. 14, 2010, now U.S. Pat. No.8,024,377, issued Sep. 20, 2011, which is a continuation of U.S. Pat.No. 7,774,308, issued Aug. 10, 2010, which claims priority to U.S.Provisional Patent Application Ser. No. 60/636,822, filed Dec. 15, 2004,each of which is incorporated herein in its entirety by this referencethereto.

BACKGROUND OF THE INVENTION

1. Technical Field

The invention relates to the storage of information. More particularly,the invention relates to an anti-item for deletion of content in adistributed datastore.

2. Description of the Prior Art

In recent years, the need for reliable and efficient storage of largeamounts of complex data has increased dramatically. Indeed, the need forextensive storage seems to have outpaced the remarkable advances made instorage technology, e.g. the increasing capacity and decreasing cost ofhard disks. As a result, storage of truly large amounts of data on asingle server is in many cases impractical, and a distributed approachis desirable. Moreover, even in those cases where storage can be handledby a single machine, a distributed datastore may offer superiorreliability and more efficient load handling.

In many distributed datastores, a single data item may be stored in oneor more locations, i.e. on more than one server. In some such systems,the location(s) at which a data item is stored is determined by aninsertion procedure in which the inserted item is propagated through anetwork of servers to the one or more servers that eventually store thedata. Such an insertion procedure presents challenges if the need fordeletion of data items arises because it may be inconvenient orcomputationally expensive to determine directly the locations at which aparticular data item is stored.

It would be advantageous to provide an efficient apparatus and methodfor deleting items within a distributed datastore. In particular, itwould be advantageous to provide an apparatus and method of deletionthat uses an already well defined procedure for insertion of data.

SUMMARY OF THE INVENTION

The invention provides an efficient apparatus and method for deletingitems within a distributed datastore. In particular, the inventionprovides an apparatus and method of deletion that uses an already welldefined procedure for insertion of data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block schematic diagram showing an exemplary distributeddatastore according to the invention;

FIG. 2 is a block schematic diagram showing superior and inferior nodes,and a sense of upstream and downstream according to the invention;

FIG. 3 is a block schematic diagram showing hosting ranges of the nodesaccording to the invention;

FIG. 4 is a block schematic diagram showing coverages of the nodesaccording to the invention;

FIG. 5 is a block schematic diagram showing insertion of a tupleaccording to the invention; and

FIG. 6 is a block schematic diagram showing insertion of an anti-tuple,i.e. deletion of a tuple, according to the invention.

DETAILED DESCRIPTION OF THE INVENTION

The invention facilitates the deletion of data items from a distributeddatastore via the addition of anti-items using a standard data insertionprocedure. Successful operation of the invention is possible in adatastore where:

-   -   The data insertion procedure is based upon the data item.        Specifically, the location at which an inserted data item is        ultimately stored is preferably consistent and is preferably        based upon the properties of the inserted data item itself; and    -   The data insertion procedure is essentially constant over time.        Specifically, the result of the data insertion procedure, i.e.        the location(s) to which the inserted item is propagated, is        preferably the same when the anti-item is inserted as it was        when the original data item was inserted. For purposes of this        disclosure, the data insertion procedure also comprises any        variant of the term “procedure”, such as a process, algorithm,        module, or mechanism. This understanding of the term “procedure”        applies likewise to the deletion procedure discussed below.

The structure of an anti-item is the same as that of a conventional dataitem, but for a single tag, flag, or bit which indicates that it is ananti-item. Importantly, this additional tag does not affect the locationat which a data item or anti-item is ultimately stored within thedatastore. These properties ensure that, upon insertion, the anti-itemreaches the same location or locations within the datastore as theoriginal data item. After the anti-item has reached these locations, thedata item is considered deleted. Deletion may entail the actual removalof both the data item and the anti-item from the datastore. Suchannihilations may be executed upon arrival of the anti-item at thestorage location or through a periodic, off-line garbage collectionprocess. Alternatively, the anti-item may be forever stored alongsidethe data item. In this case, the presence of the anti-item alongside thedata item excludes the data item from subsequent data retrievaloperations.

Exemplary Distributed Datastore

The invention is more clearly understood through consideration of anexemplary distributed data-store that satisfies the abovecharacteristics.

Datastore Structure

The exemplary datastore consists of multiple nodes connected in adirected, acyclic graph, as shown in FIG. 1, in which fourinterconnected nodes (Na, Nb, Nc, and Nd) are illustrated. Storage ofdata occurs at nodes, which are preferably servers with caching or harddisk data storage capabilities, or a combination thereof.

A particular installation of the datastore is characterized by a keylist containing N keys. Each key within the key list is defined by anordered list of values vi and an integer k. The integer represents theposition of the key within the key list. A key may thus be expressed as

Kk={v1, v2, . . . , vi, . . . , vMk}.  (1)

Here, M denotes the number of values within the key. In many instances,the ordered list of values is simply a range of integers. For example, akey may be the defined as the integers between 1 and 10, inclusive. Thatis,

Kx=[1, 10].  (2)

Data items within the datastore take the form of tuples of length N,expressed as

I=(k1, k2, . . . , kj, . . . , kN),  (3)

where kj are key values, selected from the jth key within the key list.Thus, for a key list S defined as

S={K1, K2}={[2, 7], [4, 13]},  (4)

and one possible tuple is

I=(3, 8).  (5)

A key range is a contiguous subset of the indices into the ordered listof values that define a key. A key range is expressed as a pair ofintegers representing the ends of the range, inclusive. That is,

r=[imin, imax].  (6)

The storage duties of each node within the datastore are defined by ahosting range. A hosting range is a set of N key ranges that defines asubset of all tuples that can be expressed using the key list. Forexample, for the key list of Equation (4), the hosting range

H={[2, 3], [9, 10]}  (7)

indicates that the node is responsible for storing tuples [2, 9], [2,10], [3, 9], and [3, 10]. As noted, the storage duties may be fulfilledusing a combination of caching and disk storage. Alternatively, and moreconveniently, a hosting range may be expressed using a set of tuplesdefining the corners of the covered region. For example, the hostingrange of Equation (7) may be expressed as

H={(2, 9), (3, 10)}.  (8)

As illustrated in FIG. 2, nodes may be characterized by a rank. Nodeshaving a lower rank are termed inferior, and nodes having a higher rankare termed superior. Nodes may only seek data from superior nodes, asindicated by the direction of the arrows. That is, requests for data maybe passed to superior nodes but may not be passed to inferior nodes.

The coverage of a node is the union of the tuples within its hostingrange and the tuples within the hosting ranges of the superior nodes towhich it is connected. To facilitate this calculation, a node advertisesits coverage to the inferior nodes to which it is connected. In thesimplest embodiments of the invention, an administrator configures thenetwork of nodes, explicitly indicating the connectivity between nodes,and the hosting ranges of each, and this calculation must be performedonce upon system configuration. In more sophisticated distributeddatastores, though, the connectivity of nodes and the hosting ranges ofthe nodes may be adjusted dynamically in response to measured systemloads. In such systems, the advertising of coverage occurs on an ongoingbasis.

A node has full coverage when its coverage contains all tuples that maybe specified with the key list. For any tuple, a node with full coverageeither hosts the tuple itself, or is connected to a superior node thatadvertises coverage for the tuple. By definition, a node is of rank zeroif and only if it has full coverage.

FIG. 3 shows a set of possible hosting ranges for the distributeddatastore of FIG. 1 with the key list of Equation (4). The resultingcoverages are shown in FIG. 4.

Insertion Procedure

Insertion of data into the datastore is performed by a client thatconnects to the datastore. Instructions to insert data are only passedfrom inferior nodes to superior nodes. Thus, to ensure that any tuplewithin the key list can be inserted properly, it is required that theclient connect only to nodes of rank zero.

FIG. 5 shows an example of the data insertion procedure in which theclient inserts the tuple (5, 6) at node Na. Because node Na is of rankzero, the inserted tuple is within its coverage. However, the insertedtuple is not within the hosting range of node Na, so node Na does notstore the tuple locally. Node Na then considers the coverage of thesuperior nodes to which it is connected and passes the insertion requestto those superior nodes that provide coverage for the inserted tuple. Inthis case, both Nb and Nc and provide coverage for the tuple (5, 6).Node Nb receives the insertion request. The inserted tuple is within thehosting range of node Nb, so node Nb stores the tuple locally. Afterstoring the tuple locally, node Nb considers the coverage of thesuperior nodes to which it is connected. The inserted tuple is withinthe coverage of node Nc but not within the coverage of node Nd. Node Nbtherefore passes the insertion request to node Nc but not to node Nd.

Thus, node Nc receives two insertion requests for the tuple (5, 6). Uponreceipt of the first insertion request, node Nc determines that thetuple is within its hosting range. The tuple is therefore storedlocally. Upon receipt of the second insertion request, node Nc againdetermines that the tuple is within its hosting range. However, it thennotes that the tuple is already within its local storage and thereforeignores the second insertion request. Because node Nc is not connectedto any superior nodes, the insertion procedure then terminates.

Deletion Procedure

The deletion procedure of the current invention is preferablyimplemented within the exemplary datastore through the addition of adeletion key to the key list. Specifically, the key list is defined as

S={K1 , K2 , Kd}={[1, 5], [6, 15], [0, 1]}.  (9)

The deletion key Kd is added solely to differentiate tuples fromanti-tuples, i.e. items from anti-items. Tuples are of the form (a, b,0), and all anti-tuples are of the form (c, d, 1). The deletion key isnot used to determine where within the datastore a particular tuple isstored. Thus, in the current example all nodes have a hosting range ofthe form

H={(t, v, 0), (u, w, 1)}.  (10)

That is, all nodes store both tuples and anti-tuples.

As described above, anti-items may be designated in a number ofdifferent ways. The inclusion of an additional key within the key listis merely a particularly convenient and elegant way of denoting ananti-tuple in the present example. Other possibilities include appendinga flag to the data item, or modifying a value within meta-dataassociated with the data item.

FIG. 6 shows an example of a deletion operation that is intended todelete the item inserted in FIG. 5. The deletion procedure is highlyanalagous to the insertion procedure. A client connects to the only nodeof rank zero, and requests deletion of the tuple (5, 6, 0). This isaccomplished by inserting the corresponding anti-tuple (5, 6, 1). Thecoverage values shown have been altered from FIG. 5 to reflect theinclusion of the additional key within the key list. Similarly, thetuple inserted in FIG. 5 is assumed to have been of the form (5, 6, 0).

The insertion of the anti-tuple proceeds as the insertion of the tuplein FIG. 5. Thus, the anti-tuple (5, 6, 1) arrives for storage at nodesNb and Nc. Arrival of the anti-tuple at the same location as the tupleresults in deletion of the tuple. As noted above, deletion may entailthe actual removal of the tuple from storage, which may be performed inseveral ways. Most simply, upon storage of an anti-tuple the storingnode checks for the presence of the corresponding tuple. If thecorresponding tuple is present, it is removed from storage.Alternatively, the anti-tuple may be stored alongside the tuple, andoptionally removed at a later time through a garbage collection process.In this approach, or in another alternative approach in which the tupleis stored indefinitely, the presence of the anti-tuple alongside thetuple results in the exclusion of the tuple from any subsequent dataretrieval operations.

Wildcard Key Values

Finally, the specification of an anti-tuple may also allow for a wildcard or “don't care” key value for one or more of the keys. For example,the insertion procedure propagates the anti-tuple (3, *, 1) to all nodeswith a hosting range encompassing a key value of 3 for the first key.Arrival at these nodes results in the deletion of all tuples of the form(3, k2, 0), where k2 is a any key value selected from the key K2. Thus,in the network of FIG. 4, the tuple is propagated to and delete anymatching tuples stored at nodes Nb and Nd.

The symmetry of the insertion and deletion procedure suggests that awild card key value could also be used in the insertion process. Afterinsertion, a tuple containing a wild card is propagated to all nodes forwhich the explicitly specified key values are within the hosting range.Upon arrival at these hosting nodes, all tuples within the node'shosting range that match the tuple containing the wild card are stored.This operation results in a broad insertion.

Although the invention is described herein with reference to thepreferred embodiment, one skilled in the art will readily appreciatethat other applications may be substituted for those set forth hereinwithout departing from the spirit and scope of the present invention.For example, the various operations and procedures disclosed herein maybe effected by any of software, hardware, and combinations thereof.

Accordingly, the invention should only be limited by the Claims includedbelow.

1. A method of excluding a first item from a retrieval operation in adistributed datastore, said method comprising: inserting a second iteminto said distributed datastore, wherein said first item was previouslyinserted into said distributed datastore; wherein said first item andsaid second item each comprise a plurality of key values, one key valuefor each of a plurality of keys, wherein said retrieval operation isspecified using a set of keys from said plurality of keys; wherein saidfirst item and said second item have equal key values for each keywithin said set of keys; and wherein insertion of said second itemwithin said distributed data-store excludes said first item from saidretrieval operation.
 2. The method of claim 1, wherein said plurality ofkeys comprises a binary indicator in the form of a deletion key, whereina key value of said deletion key distinguishes said first item and saidsecond item from one another, and wherein said deletion key is notwithin said set of keys.
 3. The method of claim 1, additionallycomprising the step of: removing said first item and said second itemfrom said distributed datastore immediately upon insertion of saidsecond item into said distributed datastore.
 4. The method of claim 1,additionally comprising the step of: executing an offline garbagecollection process to remove said first item and said second item fromsaid distributed datastore substantially after insertion of said seconditem into said distributed datastore.
 5. The method of claim 1, whereinsaid plurality of key values are numeric.
 6. The method of claim 1,wherein said second item comprises a wild card key value correspondingto any and all valid key values.